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CN , Abstract 

C ' We propose and study quantitative measures of smoothness / i— >■ A{f) which are adapted to 

anisotropic features such as edges in images or shocks in PDE's. These quantities govern the rate of 
approximation by adaptive finite elements, when no constraint is imposed on the aspect ratio of the 
triangles, the simplest example being Ap{f) — \\^/\det{(Pj)\\\L^ which appears when approximating 
in the L'' norm by piecewise linear elements when ^ = p~^^- quantities A{f) are not semi-norms, 
and therefore cannot be used to define linear function spaces. We show that these quantities can be 
well defined by mollification when / has jump discontinuities along piecewise smooth curves. This 
motivates for using them in image processing as an alternative to the frequently used total variation 
^ I semi-norm which does not account for the smoothness of the edges. 

> 

1 Introduction 

There exists various ways of measuring the smoothness of functions on a domain C H'^, generally 
through the definition of an appropriate smoothness space. Classical instances are Sobolev, Holder and 
Besov spaces. Such spaces are of common use when describing the regularity of solutions to partial 
04 ■ differential equations. From a numerical perspective, they are also useful in order to sharply characterize 

CO ' at which rate a function / may be approximated by simpler functions such as Fourier series, finite 

elements, splines or wavelets (see [6l[T2l|9] for surveys on such results). 

Functions arising in concrete applications may have inhomogeneous smoothness properties, in the sense 
that they exhibit area of smoothness separated by localized discontinuities. Two typical instances are 
(i) edge in functions representing real images and (ii) shock profiles in solutions to non-linear hyperbolic 
PDE's. The smoothness space that is best taylored to take such features into account is the space BV{il) 
of bounded variation functions. This space consists of those / in L^{H) such that V/ is a bounded 
measure, i.e. such that their total variation 



< 



X 



TV(/) = \f\BV :=max|^^/div(^) ; ^eViil)", Ml^ < 1 

is finite. Functions of bounded variation are allowed to have jump discontinuities along hypersurfaces of 
finite measure. In particular, the characteristic function of a smooth subdomain D d D, has finite total 
variation equal to the d — 1-dimensional Hausdorff measure of its boundary: 

\Xd\bv ^na-iidD). (1.1) 

It is well known that BV is a regularity space for certain hyperbolic conservation laws |141 117j . in the 
sense that the total variation of their solutions remains finite for all time t > 0. This space also plays an 
important role in image processing since the seminal paper |13j . Here, a small total variation is used as a 
prior to describe the mathematical properties of "plausible images" , when trying to restore an unknown 
image / from an observation h — Tf + e where T is a known operator and e a measurement noise of 
norm ||e||^2 < s. The restored image is then defined as the solution to the minimization problem 

mm{\g\Bv;\\Tg~h\\L2<e}. (1.2) 

geBV 
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From the point of view of approximation theory, it was shown in [8] that the space BV is almost charac- 
terized by expansions in wavelet bases. For example, in dimension d = 2, if / = ^ dxipx is an expansion 
in a tensor-product i^-orthonormal wavelet basis, one has 



^ f e BV ^ (dx) ewi^, 



where is the space of weakly summable sequences. The fact that the wavelet coefficients of a _BT^ 
function are weakly summable implies the convergence estimate 



where /at is the nonlinear approximation of / obtained by retaining the N largest coefficients in its 
wavelet expansion. Such approximation results have been further used in order to justify the performance 
of compression or denoising algorithms based on wavelet thresholding [7| . 

In recent years, it has been observed that the space BV (and more generally classical smoothness 
spaces) do not provide a fully satisfactory description of piecewise smooth functions arising in the above 
mentioned applications. Indeed, formula reveals that the total variation only takes into account the 
size of the sets of discontinuities and not their geometric smoothness. In image processing, this means 
that the set of bounded variation images does not make the distinction between smooth and non-smooth 
edges as long as they have finite length. 

The fact that edges have some geometric smoothness can be exploited in order to study approximation 
procedures which outperform wavelet thresholding in terms of convergence rates. For instance, it is easy 
to prove that if / = Xd where D is a bidimensional domain with smooth boundary, one can find a 
sequence of triangulations T/v with N triangles such that the convergence estimate 



holds, where Ij- denotes the piecewise linear interpolation operator on a triangulation T. Other methods 
are based on thresholding a decomposition of the function in bases or frames which differ from classical 
wavelets, see e.g. [3l[T6j[l]. These methods also yield improvements over (jl.Sp similar to (|1.4|) . The 
common feature in all these approaches is that they achieve anisotropic refinement near the edges. For 
example, in order to obtain the estimate (jl.4l) , the triangulation T/v should include a thin layer of triangles 
which approximates the boundary dD. These triangles typically have size N~'^ in the normal direction 
to dD and N^^ in the tangential direction, and are therefore highly anisotropic. 

Intuitively, these methods are well adapted to functions which have anisotropic smoothness properties 
in the sense that their local variation is significantly stronger in one direction. Such properties are not 
well described by classical smoothness spaces such as BV, and a natural question to ask is therefore: 

What type of smoothness properties govern the convergence rate of anisotropic refinement methods and 
how can one quantify these properties ? 

The goal of this paper is to answer this question, by proposing and studying measures of smoothness 
which are suggested by recent results on anisotropic finite element approximation [3J [S] . Before going fur- 
ther, let us mention several existing approaches which have been developed for describing and quantifying 
anisotropic smoothness, and explain their limitations. 

1. The so-called mixed smoothness classes have been introduced and studied in order to describe 
functions which have a different order of smoothness in each coordinate, see e.g. [20l [23]. These 
spaces are therefore not adapted to our present goal since the anisotropic smoothness that we want 
to describe may have preferred directions that are not aligned with the coordinate axes and that 
may vary from one point to another (for example an image with a curved edge). 

2. Anisotropic smoothness spaces with more general and locally varying directions have been investi- 
gated in [15) . Yet, in such spaces the amount of smoothness in different directions at each point 
is still fixed in advance and therefore again not adapted to our goal, since this amount may differ 
from one function to another (for example two images with edges located at different positions). 



\\f-fN\\L-<CN-'/^\f\BV, 



(1.3) 



\\f-lT.f\\L^<CN-^ 



(1.4) 
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3. A class of functions which is often used to study the convergence properties of anisotropic approx- 
imation methods is the family of C" — C" cartoon images, i.e. functions which are C™ smooth on 
a finite number of subdomains ,fc separated by a union of discontinuity curves {T 

that are C" smooth. The defects of this class are revealed when searching for simple expression 
that quantifies the amount of smoothness in this sense. A natural choice is to take the supremum 
of all C™(ili) norms of / and C" norms of the normal parametrization of Tj. We then observe that 
this quantity is unstable in the sense that it becomes extremely large for blurry images obtained by 
convolving a cartoon image by a mollifier ^ps = ■^fiji) as (5 — > 0. In addition, this quantity does 
not control the number of subdomains in the partition. 

4. A recent approach proposed in [10] defines anisotropic smoothness through the geometric smooth- 
ness properties of the level sets of the function /. In this approach the measure of smoothness is not 
simple to compute directly from / since it involves each of its level sets and a smoothness measure 
of their local parametrization. 

The results of [H [5l [19] describe the L^'-error of piecewise linear interpolation by an optimally adapted 
triangulation of at most N elements, when / is a function of two variables. This error is defined as 

<^N{f)p-= inf Wf-IrfWLp- 
#{T}<N 

It is shown in for p = cx3 and in [19] for all 1 < p < cjo that that 

limsup NaNif)p<CApif), (1.5) 

Af->-|-oo 

where C is an absolute constant and 



Apif) :=||v/|det(d2/)||U., - := 1 + -. (1.6) 

T p 

Moreover, this estimate is known to be optimal in the sense that liminfAr_i.-j,oo NaNif)p ^ cylp(/) also 
holds, under some mild restriction on the class of triangulations in which one selects the optimal one. 
These results are extended in [19 to the case of higher order finite elements and space dimension d > 2, for 
which one can identify similar measures / i— > A{f) governing the convergence estimate. Such quantities 
thus constitute natural candidates to measure anisotropic smoothness properties. Note that Ap(f) is 
not a semi-norm due to the presence of the determinant in (|1.6p . and in particular the quasi-triangle 
inequality Ap{f + g) < C{Ap{f ) + Ap{g)) does not hold even with C > 1. 

Our paper is organized as follows. We begin in §2 by a brief account of the available estimates on 
anisotropic finite element interpolation, and we recall in particular the argument that leads to (|1.5p with 
the quantity Ap{f) defined by (|1.6p . 

Since Ap{f) is not a norm, we cannot associate a linear smoothness space to it by a standard completion 
process. We are thus facing a difficulty in extending the definition of Ap(f) to functions which are not 
C^-smooth and in particular to cartoon images such as in item 3 above. Since we know from ()1.4p that 
for such cartoon images the error of adaptive piecewise linear interpolation decays like iV~^, we would 
expect that the quantity 

.42(/) = ||V|det(d2/)lllLV3, 

corresponding to the case p — 2 can be properly defined for piecewise smooth functions. We address this 
difficulty in §3 by a regularization process: if / is a cartoon image we introduce its regularized version 

fs-.^f^ifs, (1-7) 

where (ps — j2'p{ji) is a standard mollifier. Our main result is the following; for any cartoon image / of 
(j2 _ (j2 lyTpe, the quantity A2{fs) remains uniformly bounded as (5 — > and one has 




3 



where [/](s) and k(s) respectively denote the jump of / and the curvature of Fj at the point s, and where 
C{(p) is a constant that only depends on the choice of the moUifier. This constant can be shown to be 
uniformly bounded by below for the class of radially decreasing mollifiers. This result reveals that ^2(/) 
is stable under regularization of cartoon images (in contrast to the measure of smoothness described in 
item 3 above). We also discuss the behaviour of Ap when p 7^ 2. 

These results lead us in §4 to a comparison between the quantity A2{f) and the total variation TV(/). 
We also make some remarks on the existing links between the limit expression in ()1.8|) and classical results 
on adaptive approximation of curves, as well as with operators of affine-invariant image processing which 
also involve the power 1/3 of the curvature. 

We devote §5 to numerical tests performed on cartoon images that illustrate the validity of our results 
and we describe in §6 the extension of our results to finite elements of higher degree and higher space 
dimensions. Concluding remarks and perspectives of our work are given in §7. 

2 Anisotropic finite element approximation 

A standard estimate in finite element approximation states that if / £ W'^'^{ft) then 

\\f-lTj\\LP<Ch^d'f\\L., 

where T/i is a triangulation of mesh size h :— max-rgTh diam(T). If we restrict our attention to a family 
quasi-uniform triangulations, h is linked with the complexity N := H^{Th) according to 

Cih-^ <N < C2h'^ 

Therefore, denoting by cJ^"*(/)lp the approximation by quasi-uniform triangulations of cardinality 
N, we can re-express the above estimate as 

ar'(/)Lp < CN-'\\d^f\\L.. (2.9) 

In order to explain how this estimate can be improved when using adaptive partitions, we first give some 
heuristic arguments which are based on the assumption that on each triangle T the relative variation 
of (Pf is small so that it can be considered as constant over T, which means that / coincides with a 
quadratic function qt on each T. Denoting by It the local interpolation operator on a triangle T and by 
^T{f)p ■= 11/ — ^t/|1lp(t) the local error, we thus have according to this heuristics 

Ter Ter 

We are thus led to study the local interpolation error eT{q)p when q G P2 is a a quadratic polynomial. 
Denoting by q the homogeneous part of g, we remark that 

e-T{q)p = eT(q)p. 

We optimize the shape of T with respect to the quadratic form q by introducing a function Kp defined 
on the space of quadratic forms by 

Kpi^) '■= |^nf^eT(q)p, 

where the infimum is taken among all triangles of area 1. It is easily seen that eT(q)p is invariant by 
translation of T and so is therefore the minimizing triangle if it exists. By homogeneity, it is also easily 
seen that ^ ^ 

inf eT(q)p = a"ifp(q), - := - + 1, 

\T\=a T p 

and that the minimizing triangle of area a is obtained by rescaling the minimizing triangle of area 1 if it 
exists. Finally, it is easily seen that if ip is an invertible linear transform 

Kp{(ioip) = \det{ip)\Kp{(i), 
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and that the minimizing triangle of area |det((/3)|~^ for q o is obtained by apphcation of ip~^ to the 
minimizing triangle of area 1 for q if it exists. If det(q) ^ 0, there exists a ip such that q o is either 
up to a sign change, and we have |det(q)| = |det(93)|^^. It follows that Kp{q) has the 



y or X 



simple form 



if,(q) =a|det(q)|i/2, 



(2.10) 



where cr is a constant equal to Kp{x^ + y"^) if det(q) > and to Kp{x'^ — y^) if det(q) < 0. One easily 
checks that this equality also holds when det(q) = in which case i^p(q) = 0. 

Assuming that the triangulation T is such that all its triangles T have optimized shape in the above 
sense with respect to the quadratic form q^ associated with qx, we thus have for any triangle T G T 



eTif)p = eTMp^\T\-Kp{qT) 



L-{T) 



since we have assumed = qr on T. In order to optimize the trade-off between the global error and 
the complexity N = #(7^), we apply the principle of error equidistribution: the triangles T have area 
such that all errors eT(qT)p are equal i.e. eT(qT)p = V for some 77 > independent of T. It follows that 
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and therefore 

<^N{f)p<\\f- 

which according to (|2.10p implies 



Am 



N' 



(2.11) 



with Ap defined as in (|1.6p . 

The estimate (|2.1ip is too optimistic to be correct: if / is a univariate function then Ap{f) — 
while (JN{f)p niay not vanish. In a rigorous derivation such as in [2], one observes that if / G C^, the 
replacement of (P f hy a constant over T induces an error which becomes negligible only when the triangles 
are sufficiently small, and therefore a correct statement is that for any e > there exists A'o — NQ{f,e) 
such that 



<^N{f)p < N- 



2 



for all N >No, i.e. 



limsup AfcrAr(/)p < 



(2.12) 



(2.13) 



which according to (|2.10p implies (jl.Sp . 



3 Piecewise smooth functions and images 

As already observed, the quantities Ap{f) are well defined for functions f E C^, but we expect that they 
should in some sense also be well defined for functions representing — "cartoon images" when 
p < 2. We first give a precise definition of such functions. 

Definition 3.1 ^ cartoon function on an open set fl is a function almost everywhere of the form 

l<i<k 

where the Hi are disjoint open sets with piecewise boundary, no cusps (i.e. satisfying an interior and 
exterior cone condition), and such that Vl — U^Lj^rJ;, and where for each 1 < i < k, the function fi is 
on 
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Let us consider a fixed cartoon function / on an open polygonal domain n (i.e. 51 is such that SI is a 
closed polygon) associated with a decomposition {ili)i<i<k- We define 

r:= y an,, (3.14) 

l<i<k 

the union of the boundaries of the Sl^. The above definition implies that F is the disjoint union of a finite 
set of points V and a finite number of open curves {Ti)i<i<i. 

i<i<i 

Furthermore for all 1 < i < j < we may impose that nTjCZV (this may be ensured by a splitting 
of some of the F^ if necessary). 

We now consider the piecewise linear interpolation Ij-nJ of / on a triangulation T/v of cardinality N. 
We distinguish two types of elements of T/v- A triangle T S 7jv is called "regular" if T n F = 0, and we 
denote the set of such triangles by T^. Other triangles are called "edgy" and their set is denoted by 7^. 
We can thus split ft according to 

n := (UTer^T) U (Utst^T) = 17^ U nj,. 

We split accordingly the interpolation error into 

\\f - It. f WUn) ^ f If-lT.fl'+f l/-/r„/r 

We may use 0{N) triangles in TJ^ and 7^ (for example N/2 in each set). Since / has discontinuities 
along F, the L°° interpolation error on does not tend to zero and should be chosen so that 
has the aspect of a thin layer around F. Since F is a finite union of curves, we can build this layer 
of width 0(iV~^) and therefore of global area |S7^| < CN^"^ , by choosing long and thin triangles in TJ^. 
On the other hand, since / is uniformly on f2^, we may choose all triangles in TJ^ of regular shape 
and diameter hx < CN^^I"^ . Hence we obtain the following heuristic error estimate, for a well designed 
anisotropic triangulation: 



\\^-lrJ\Wi-^) = (ll/-/r„/ll!.(n^) + ll/-/r„/lli.(^^)) 



i/p 



N 



1/p 



< (11/ - iT.fWU^nJ^'^l + 11/ - iT.fWU^n^jn^ 

< C{N-P + N-^f/P, 

and therefore 

where the constant C depends on ||'i^/||Loo(n\r): ||/||L°°(n) and on the number, length and maximal 
curvature of the curves which constitute F. 

Observe in particular that the error is dominated by the edge term ||/ — Itn f\\Lp(n%) when p > 2 and 
by the smooth term ||/ — -^Tjv/llLp(n^) when p < 2. For the critical value p — 2 the two terms have the 
same order. 

For p < 2, we obtain the approximation rate which suggests that approximation results such 
as (jl.Sp should also apply to cartoon functions and that the quantity Ap{f) should be finite. We would 
therefore like to bridge the gap between anisotropic approximation of cartoon functions and smooth 
functions. For this purpose, we first need to give a proper meaning to Ap(f) when / is a cartoon 
function. This is not straightforward, due to the fact that the product of two distributions has no meaning 
in general. Therefore, we cannot define det{(P f) in a distributional sense, when the coefficients of (P f are 
distributions without sufficient smoothness. Our approach will rather be based on regularisation. This 
is additionally justified by the fact that sharp curves of discontinuity are a mathematical idealisation. In 
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real world applications, such as photography, several physical limitations (depth of field, optical blurring) 
impose a certain level of blur on the edges. 

In the following, we consider a fixed radial nonnegative function of unit integral and supported in 
the unit ball, and we define for all (5 > and / defined on il, 

<^5(z) ^<y3 (I) and/5 = /*(^5. (3.16) 

Our main result gives a meaning to Ap{f) based on this regularization. If / is a cartoon function on a 
set {I, and ii x £T\V, we denote by [f]{x) the jump of / at this point. We also denote t{x) and 11(2:) 
the unit tangent and normal vectors to F at x oriented in such way that det(t, n) = +1, and by k{x) the 
curvature at x which is defined by the relation 

dt(x)^{x) = K{x)n{x). 

For p G [1, 00] and r defined by ^ := 1 + we introduce the two quantities 



Spif) ||v/|det(d2/)lliLMn\r) =^p(/|o\r), 

Epif) llv^milLMD, 

which respectively measure the "smooth part" and the "edge part" of /. We also introduce the constant 

Cp,^ ||\/|$$'|||l-(r), where $(.t) := / ip{x,y)dy. (3.17) 

Note that fs is only properly defined on the set 

■~{z(^VL; B{z,S) C n}, 



and therefore, we define Ap{fs) as the L'^ norm of ^/\det{tPfs)\ on this set. 
Theorem 3.2 For all cartoon functions f , the quantity Ap{fs) behaves as follows: 
• If p < 2, then 

lim Apifs)^Sp{f). 



Ifp~ 2, then t = ^ and 



frnA^ifs) = {S2{fr + E,{frCl^y. (3.18) 



If p > 2, then Ap{fs) — )■ 00 according to 



lim 5^-pAp{fs) = Ep{f )Cp,^. (3.19) 

(5— >0 

Remark 3.3 This theorem reveals that as (5 — > 0, the contribution of the neighbourhood ofT to Ap{fs) is 
negligible when p < 2 and dominant when p > 2, which was already remarked in the heuristic computation 
leading to (|3.15p . 

Remark 3.4 It seems to be possible to eliminate the "no cusps" condition in the definition of cartoon 
functions, while still retaining the validity of this theorem. It also seems possible to take the more natural 
choice (p{z) = ;^e~"^" , which is not compactly supported. However, both require higher technicality in 
the proof which we avoid here. 

Before attacking the proof of Theorem 13. 2[ we show below that the constant Cp,^p involved in the 
result for p > 2 is uniformly bounded by below for a mild class of moUifiers. 
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Proposition 3.5 Let if be a radial and positive function supported on the unit ball such that J ip ~ 1 
and that ip{x) decreases as \x\ increases. For any p >2 we have 



TT \T + Z 

and this lower bound is optimal. There is no such bound if p < 2, but note that Theorem \3.2\ does not 
involve Cp^tp for p < 2. 

Proof: Let D be the unit disc of . We define a non smooth molUfier tp and a function 5* as follows 



tp := — ^ and "^{x) 

TT 



ip{x,y)dy. 



One eas: 



ily obtains that = - x^X[-i.i]ix) and '^'{x) = — ^^=X[-i,i] (x), hence 



For all (5 > we define ips ■= <5 ■^'0((5 ^•), and 'i'six) := Jg^4's{x,y)dy. Similarly we obtain 



4a;. 



Hence 
Note that 



^sixWsix)] 



2 



\x\ 2 dx 



2 /2^i+i\" 2 



-5 



7r(52 V f + 1 



TT V T + 2 



If p > 2 and (5 G (0, 1] then C^^^^ > Cp^^ = — 



n \ t + 2 



(3.20) 



The moUifier ip of interest is radially decreasing, has unit integral and is supported on the unit ball. It 
follows that there exists a Lebesgue measure fj, on (0, 1], of mass 1, such that 

ip =^ tps dfi{S). 
Jo 

Hence ^{x) :— J^ip{x,y)dy = 'i'six) d^{5), for any a; G M. Since s M- s"^ is concave on R+ when 
< r < 1, we obtain 

^{xY =( [ ^s{x) d^iiS)] > [ ^s{xy dfi{5) 



Similarly, since the sign of 5'^ (x) is independent of (5, I $'(a:) I = (^Jq l^^'si^M dfi{S)^ > |*^(a:)|^ cf/i(5). 
Applying the Cauchy-Schwartz inequality we obtain 



Eventually we conclude using the previous equation and p.20p as follows 



which concludes the proof of this lemma. 



8 



The rest of this section is devoted to the proof of Theorem 13.21 Since it is rather involved, we spht 
its presentation into several main steps. 



Step 1: decomposition of Ap{fs). Using the notation K{M) y/ \ dct M \ , we can write 

ApifsY - / Kid'fsY. (3.21) 
We decompose this quantity based on a partition of il^ into three subsets 

The first set fl^ corresponds to the smooth part: 

fls :— (J where il^ :— {z G Qi ; d{z, ft \ Qi) > S}. 

l<i<k 

Note that fig is strictly contained in . The second set corresponds to the edge part: we first define 
ro U where F^"^, := {z e ; d{z,T\T,) > 2,5}, 

i<3<l 

and then set 

Ts IJ Tj,, where r^-, :^ {z e n ; d(z,T) < 6 and nriz) G F^^J 

i<]<i 

where Trr{z) denotes the point of F which is the closest to z. The third set corresponds the corner part: 

Vs :=n^\insurs). 

The measures of the sets Tg and Vs tends to as (5 — 0, while \Hs\ tends to More precisely, we have 

IF5I < CS and mi < C6^ 

where the last estimate exploits the "no cusps" property of the cartoon function. We analyze separately 
the contributions of these three sets to p.2ip . 

Step 2: Contribution of the smooth part fts. The contribution of fig to the integral p.2ip is 
easily measured. Indeed, let us define 



Qs{z) := 



Kid^fsiz)r iizens, 

otherwise. 



then we have pointwise convergence Qs{z) "> K{d'^ f{z)Y on 57 \ F. Since the ^-neighbourhood of Vlg is 
included in ri \ F, we have 

\\d^{f * f5)\\L^(n,) = ||(c^^/) * fs\\L^(n,) < f\\L--(n\T)\Ws\\L^ = ||d^/||L-.(n\r) 
Since K is 1-homogeneous and continuous, we have 

K{d''fs) < CK\\d^f\\L^^n\r), Ck := max K{M), 
and we conclude by dominated convergence that 

lim / K{d^fsY = lim / Qs = I K{d^fY. 

Step 3: Contribution of the corner part Vs- We only need a rough upper estimate of the contribution 
of Vs to the integral (I3.21[) . We observe that 

M 

\\d\f * 'fs)\\L^in) = 11/ * (rfV5)llL-(0) < ll/llL~(n)l|rf'<^5|Ui(R2) = ^, 



where M ||/||ioo(-j-2)||(i^(y5||^i(i{2'). It follows that 

K{d?hY < \Vs\ [ck < CS'-'\ 

If T < 1 , this quantity tends to and is therefore negligible compared to the contribution of the smooth 
part. If r = 1, which corresponds to p = oo, our further analysis shows that the contribution of the edge 
part tends to +oo, and therefore the contribution of the corner part is always negligible. 



Step 4: Contribution of the edge part Ts- This step is the main difficulty of the proof. We 
make a key use of an asymptotic analysis of fs on Ts, which relates its second derivatives to the jump 
[/] and the curvature k as (5 — > 0. We first define for all (5 > the map 

Us -.TXv X [-1,1] n 

(x.u) H- 5- x + Sun{x). 

We notice that according to our definitions, for S small enough, the map Us induces a diffeomorphism 
between Tg x [—1, 1] and Ts, such that nr{Us{x,u)) = x and d{Us{x,u),T) — \Us{x,u) — x\ = S\u\. We 
establish asymptotic estimates on the second derivatives of fs which have the following form: 



a„,„/5(z)-^[/](x)<f'(w) 



dt.tfsiz) + -[f]{x)^ix)1>{u) 



C 

< — 

- 5 



< 



< 



C 
J 



(3.22) 
(3.23) 
(3.24) 



where lims^o'^iS) — and with the notation z = Us{x,u). The constant C and the function lj depend 
only on /. The proof of these estimates is given in the appendix. As an immediate consequence, we 
obtain an asymptotic estimate of K{cffs) — y/\ det{(Pfs)\ of the form 



S^K{d'fs{z)) - ^\4x)\mx)Wmu)^'iu)\ <uiS), 
where lim5_>.o w((5) — 0, and the function uj depends only on /. Using the notations 



(3.25) 



we thus have 



gsiz) S^Kid'fsiz)), \{x) ^^|[/](a;)|, ^i{u) ^Mu)<i>' {u)\, 



\gsiz) - X{x)^l{u)\ < uj{S), 



(3.26) 

for all X G r°, u e [—1,1] and 5 > sufficiently small, with z — Us{x,u). We claim that for any 
continuous functions {gs,\,fJ.) satisfying (I3.26|) . we have for any r > 0, 



lim S ^ 

5^0 



9s 



\[xY dx j fi{uydu, 



which is in our case equivalent to the estimate 

iim<5i--i / K{d^fsY= [ imri^r/^ / m 



nr/2 



(3.27) 



(3.28) 



In order to prove (|3.27p . we may assume without loss of generality that t = 1 up to replacing (gs, A,/i) 
by (55 , A"^, ^■^). We first express the jacobian matrix of Us using the bases Bi — ((t(x), 0), (0, 1)) and 
B2 — (t(x), n(a;)) for the tangent spaces of F x [—1, 1] and ft. This gives the expression 



[dUsix,u)]Bi 



B2 



1 — Suk{x) 
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and therefore | dei{dUs{x,u))\ — S — S'^uk{x). Since Bi and B2 are orthonormal bases, this quantity is 
the jabobian of Us at {x, w), and therefore 



gs = S / gs{x + Sun{x)){l — 5uK{x))dx du 

ir«x[-i,i] 

Combining with (13.26^ . and using dominated convergence we obtain (j3.27l) . 

Step 5: summation of the different contributions. Summing up the contributions of il^, Vs 
and r^, we reach the estimate 

/ Kid'fsY = f K{d^UY + ! Kid^fsY + f Kid'fsY 
Jn Jns Jvs JTs 

= (I K{d'fY + ) + B{s)s^-'^ + 51-^ ( I / m'v'^ + e^is)] , 

\Jn\r I \Jr Jm J 

= + em + B{b)b^-^^ + d'-^-{E,ifYCl^ + £2(<5)), 



where lims^a ei{S) = hma_>.o £2 ((5) = and B{6) is uniformly bounded. This concludes the proof of 
Theorem [121 



4 Relation with other works 

Theorem 13. 21 allows us to extend the definition of ^2(/) when / is a cartoon function, according to 

A2{f) {S2{ff/' + E2{fY/'C'2!^y^\ (4.29) 



We first compare this additive form with the total variation TV(/). If / is a cartoon function, its total 
variation has the additive form 

TV(/):=|lV/|Ui(o\r) + limilLi(r), (4.30) 

Both ()4.29p and (|4.30p include a "smooth term" and an "edge term". It is interesting to compare the 
edge term of A2{f), which is given by 



E2{f) = \Wm]\\LV^ir), 

up to the multiplicative constant C2^(p, with the one of TV(/) which is simply the integral of the jump 

Jif) l|[/]||LHr), 

Both terms are 1-homogeneous with the value of the jump of the function /. In particular, if the value 
of this jump is 1 (for example when / is the characteristic function of a set of boundary F), we have 



E2 



if) -{I (4.31) 



while J(/) coincides with the length of F. In summary, A2 (/) takes into account the smoothness of edges, 
through their curvature k, while TV(/) only takes into account their length. 

Let us now investigate more closely the measure of smoothness of edges which is incorporated in 
A2{f). According to (|4.3ip . this smoothness is meant in the sense that the arc length parametrizations 
of the curves that constitute F admit second order derivatives in Ls. In the following, we show that 
this particular measure of smoothness is naturally related to some known results in two different areas: 
adaptive approximation of curves and affine-invariant image processing. 
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We first revisit the derivation of the heuristic estimate (|3.15p for the error between a cartoon function 
and its hnear interpolation on an optimahy adapted triangulation. In this computation, the contribution 
of the "edgy triangles" was estimated by the area of the layer 51^ according to 

\\f-Ir.f\\LHn^,)<\\f\\L^P'N\'^''- 

Then we invoke the fact that F is a finite union of curves Tj in order to build a layer of global area 
— CN~'^, which results in the case p — 2 into a contribution to the error of the order 0{N~-^). 
The area of the layer is indeed of the same order as the area between the edge F and its approximation 
by a polygonal line with 0{N) segments. 

Each of the curves F^ can be identified to the graph of a function in a suitable orthogonal coordinate 
system. If 7 is one of these functions, the area between F and its polygonal approximation can thus 
be locally measured by the error between the one-dimensional function 7 and a piecewise linear 
approximation of this function. Since 7 is C^, it is obvious that it can be approximated by a piecewise 
linear function on 0{N) intervals with accuracy C(iV^^) in the L°° norm and therefore in the norm. 
However, we may ask whether such a rate could be achieved under weaker conditions on the smoothness 
of 7. The answer to this question is a chapter of nonlinear approximation theory which identifies the 
exact conditions for a function 7 to be approximated at a certain rate by piecewise polynomial functions 
on adaptive one-dimensional partitions. We refer to [9] for a detailed treatment and only state the result 
which is of interest to us. We say that a function 7 defined on a bounded interval / belongs to the 
approximation space A'^iLP) if and only if there exists a sequence {pn)n>o of functions where each 
is piecewise affine on a partition of / by intervals such that 

\h-PN\\LP <CN-'. 

For < s < 2, it is known that 7 e A^iLP) provided that 7 e B^ .^{I) with i i -f s, where B^_^{I) is 

the standard Besov space that roughly describes those functions having s derivatives in L"^. In the case 

s — 2 and p = 1 which is of interest to us, we find t — ^ and therefore 7 should belong to the Besov space 

i?| 1 (/). Note that in our definition of cartoon functions, we assume much more than B\ i smoothness 

3 ' 3 3 ' 3 

on 7, and it is not clear to us if Theorem (j3.2p can be derived under this minimal smoothness assumption. 

However it is striking to see that the quantity E2{f) that is revealed by Theorem p.2|) precisely measures 

the second derivative of the arc- length parametrization of F in the norm, up to the multiplicative 

weight Let us also mention that Besov spaces have been used in [10] in order to describe the 

smoothness of functions through the regularity of their level sets. Note that edges and level sets are two 

distinct concepts, which coincide in the case of piecewise constant cartoon functions. 

The quantity is also encountered in mathematical image processing, for the design of simple 

smoothing semi-groups that respect affine invariance with respect to the image. Since these semi-groups 

should also have the property of contrast invariance, they can be defined through curve evolution operators 

acting on the level sets of the image. The simplest curve evolution operator that respects affine invariance 

is given by the equation 

f - -W"- 

where n is the outer normal, see e.g. [5). Here the value 1/3 of the exponent plays a critical role. The fact 
that we also find it in E2{f) suggests that some affine invariance property also holds for this quantity as 
well as for A2{f). We first notice that if / is a compactly supported function of two variables and T 
is a bijective affine transformation, then with / such that 

/ = /°r, 

we have the property 

cffiz) = L^d?f{Tz)L, 
where L is the linear part of T and its transpose, so that 



v/|dct(d2/W)l = I det L\ •^\dei{<Pf{Tz))\. 
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By change of variable, we thus find that 

Apif) = |deti|i/--Mp(/) = |detL|i/PAp(/). (4.32) 

A similar invariance property can be derived on the interpolation error o'jv(/)p = H/^-^Tjv/IU'' where Tzv 
is a triangulation which is optimally adapted to / in the sense of minimizing the linear interpolation error 
in the norm among all triangulations of cardinality N. We indeed remark that an optimal triangulation 
for / is then given by applying T to all elements of Tjv- For such a triangulation Tn ■— T{Tn), one has 
the commutation formula 

and therefore we obtain by a change of variable that 

^w(/)p = ll/-^r„/IU- = |cletL|i/f|l/-/r„/||LP = \det L\'/PaN{f)p. (4.33) 

Let us finally show that if / is a cartoon function, then E2{f) satisfies a similar invariance property 
corresponding to p = 2, namely 

E2{f)^\detL\'/-'E2if). (4.34) 

Note that this cannot be derived by arguing that ^2(/) satisfies this invariance property when / and / 
are smooth, since we lose the affine invariance property as we introduce the convolution by (pg: we do 
not have 

{f oT) * ips ^ {f * ips) o T. 

unless T is a rotation or a translation. 

Let Tj be one of the pieces of F and : [0, Bi] ^ a, regular parametrisation of Fj. The curvature 
of F on Tj at the point jj (t) is therefore given by 



det(7j(i),7;(t)) 



Since f ^ f ° T, the discontinuity curves of / are the images of those of / by T: 

f, =r(F,). 

The curvature of Tj at the point T{jj{t)) is therefore given by 

detiW^jt), WJjt)) ^ ^^^^ det(7K0,7;(t)) 

This leads us to the relation : 

idet(i)|V3|^(^^.(t))|i/3 = mn^^mnwm^ (4-36) 

and therefore 



2/3|~|l/3 ^ 



|[/](T(7,(t)))p/'l^^(T(7,(i)))r/' wHitm, 
"'0 

= |detL|V3|^ \m?'M'"- 
Summing over all j = 1, • • • ,1 and elevating to the 3/2 power we obtain (|4.34p . 
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Figure 1: The Logan-Shepp image (left), modified image (center), smoothed image (right). 

5 Numerical tests 

In this section, we vahdate our previous results by numerical tests applied to a simple cartoon image: the 
Logan-Shepp phantom. We use a 256 x 256 pixel version of this image, with a slight modification which 
is motivated further. This image is iteratively smoothed by the numerical scheme 



2 



8 



(5.37) 



This scheme is an explicit discretization of the heat equation. Formally, as n grows, u" is a discretization 
of 

1 IN 11^ 



with ip5{z) 



(5.38) 



where u stands for the continuous image. The determinant of the hessian is discretised by the following 
9-points formula 



2u-j- + u"j„i)(w"+ij- 



2u 
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For each value of n, we then compute the i"^ norm of the array (y^MTjl) fo^^ ^ I^' 1]' which corresponds 
to p e [1, oo] with i := 1 + i. This norm is thus a discretization of the quantity 



^\det{d^u*cp^^))\ 



For each value of n we obtain a function r e [i, 1] — > D^ir) G M+. 

As n grows, three consecutive but potentially overlapping phases appear in the behaviour of the 
functions which are illustrated on Figure 2. 

1. For small n, the 9-points discretisation is not a good approximation of the determinant of the 
hessian due to the fact that the pixel discretization is too coarse compared to the smoothing width. 
During this phase, the functions Z3„ decay rapidly for all values of t. 



2. For some range of n, the edges have been smoothed by the action of (|5.37l) . but the parameter 
Ay^ in (j5.38l) remains rather small. Our previous analysis applies and we observe that Z)„(2/3) is 
(approximately) constant while Dn{T) increases for r < 2/3 and decreases for r > 2/3. 

3. For large n, the details of the picture fade and begin to disappear. The picture begins to resemble 
a constant picture. Therefore the functions I?„ decay for all values of t, and eventually tend to 0. 
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Figure 2: The curves Dn{T) for n < 50 (left), 50 < n < 100 (center) and 100 < n < 200 (right). 

In our numerical experiments, we used the well known Shepp-Logan Phantom with a slight modi- 
fication as shown on Figure 1: we have removed the thin layer around the head, which represents the 
skull, because it disappears too quickly by the smoothing procedure and causes phases 1 and 3 to over- 
lap, masking phase 2. For more complicated functions /, such as most photographic pictures, phases 
1 and 3 also tend to overlap for similar reasons. Indeed, these pictures often have details at the pixel 
scale, including electronic noise due to the captor. Since these details disappear early phase 3 begins 
immediately, therefore phase 2 cannot be observed. 

6 Extension to higher dimensions and higher order elements 

The results on approximation by anisotropic bidimensional piecewise linear finite elements that we have 
exposed in §2 have been generalized in |19| to the case of elements of arbitrary order m — 1 defined on 
partitions of 51 C by simplices. Here, the local error is defined as 

em,T(/) 11/ - Im,.Tf\\LP(T), 

where Im.T denotes the local interpolation operator on Pm-i for a d-dimensional simplex T. This 
operator is defined by the condition 

for all points 7 G T with barycentric coordinates in the set {0, j^^, 7^^^ ' ' ' '^}- Then one defines for 
any homogeneous polynomial q G P™, 

KmA'i) ||,yl^e„^T(q)p- 

We refer to Km,p as the shape function. For piecewise linear elements in dimension two, i.e. m ~ d ^ 2, 
we have observed that Kp — if2,p has the special form given by (|2.10p which justifies the introduction of 
the quantity Ap{f). In a similar way, it can easily be proved that for piecewise linear elements in higher 
dimension, i.e. m = 2 and d > 2, one has 

ci|det(q)|l/'' < K2,p(q) < C2|det(q)|l/'^. 
For piecewise quadratic elements in dimension two, i.e. m = 3 and d = 2, it was proved in [19 that 

Ci|disc(q)|l/4 < i^3^p(q) < C2|disc(q)|l/^ 
for any homogeneous polynomial q G P3, where 

disc(ax^ + bx^y + cxy"^ + dy^) := Jd^c^ - 4ac^ - 4&^d -f 18a6cd - 21a^d^. 

For other values of m and d, equivalent expressions of ii'm,p(q) hi terms of polynomials in the coefficients 
of q are available but of less simple form. 
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Defining the finite element interpolation error by an optimally adapted partition 

^nU)p'= inf \\f - Im.rfWh'p, 

#{T)<N 

where Im,T is the global interpolation operator for the simplicial partition T, the following generalization 
of (Pl^ is proved in [H]: 



limsupiV aN{f)p < Cd 



m! ) 



1 1 m . 
, -■■=- + -7- (6-39) 

L^(o) T p a 



The constant Cd is equal to 1 when d = 2 but larger than 1 when d > 2 due to the impossibility of exactly 
tiling the space with locally optimized simplices. If / is a C™ function of d variables, it is therefore natural 
to consider the quantity 

1 1 77? 

^ ' T p a 

as a possible way to measuring anisotropic smoothness. For d — 2 and piecewise linear elements, we have 
seen in §2 that A2^p{f) is equivalent to the quantity Ap{f). 

Similarly to Ap we are interested in the possible extension of A,„ p to cartoon functions. We first 
introduce a generalisation of the notion of cartoon functions to higher piecewise smoothness tti and 
dimension d. 

Definition 6.1 Let m > 2 and d > 2 be two integers. Let CZ M."^ be an open set. We say that a function 
f defined on fl is a C™ cartoon function if it is almost everywhere of the form 



f— ^ fiXfli, 



l<i<k 



where the fli are disjoint open sets with piecewise boundary, no cusps (i.e. satisfying an interior and 



exterior cone condition), and such that Q, = U^Lj^fii. Additionally, for each I < i < k, the function fi is 
assumed to be C™ on fli. 

Let us consider a fixed cartoon function / on a polyhedral domain 17 C M"* (i.e. il is such that H. 
is a closed polyhedron), and a decomposition (rii)i<i<fc of f2 as in definition 16.11 As before we define 
r :— Ui<i<fc the union of the boundaries of the Qi. Our assumptions on the sets {fti)i<i<k imply 
that r is the union of a finite number of open hypersurfaces (rj)i<j<i, and of a set V of dimension d—2. 

As in §3, we now consider a sequence /at of piecewise linear approximations of / on simplicial partitions 
T/v of cardinality N. We distinguish two types of elements of Tn- A simplex T G Tn is called "regular" 
if T n r 0, and we denote the set of these simplices by 7^. Other simplices are called "edgy" and their 
set is denoted by We can again split il according to 

n (Urer^T) U (Urer^T) = f]^ U n%. 

Heuristically, if the partitions T/v are built with approximation error minimisation in mind, the number 
of elements should be balanced between TJ^ and T§ . The partition TJ^ tends to cover most of the surface 
of O, with simplices of diameter < CN^i , and L°° approximation error \f — /jv| < CN^'^ (since we use 
Vm-i elements). On the other hand, since / has discontinuities along F, the L°° approximation error 
on 7^ does not tend to zero, and should thus be chosen so as to produce a thin layer around F. Let 
h be the typical diameter of an element of T§. Since the Tj has bounded curvature, this layer can be 
made of width 0{h^) and therefore the layer around F has volume bounded by h^T-Ld-i{^) up to a fixed 
multiplicative constant, where Hd-i(r) is the d — 1 dimensional Hausdorff measure of F. On the other 
hand the minimal number of such elements of diameters h needed to cover F is bounded by /i^~'''Hd-i(F) 
up to a fixed multiplicative constant. Eventually, we find that the layer around F has volume bounded 
by CN-^. 

Hence we have the following heuristic error estimate, for a well designed anisotropic partition: 



1/ - /jv||LP(n) < 11/ - /jv||lp(o;,) + 11/ - /jvllLp(a^) 

< 11/ - /jv||L~(o^)|f^^ 

< C(iV"^ +7V5^T^) 



< 11/ - fNh^in^jn^Nl^ + 11/ - fN\\L-in^jn%\ 
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This leads us to define a critical exponent 



Pc = Pc{m, d) := —-: — . 

7n(d — 1) 

If one measures the error in U' norm with p > Pc{m, d), then the contribution of the edge neighbourhood 
dominates, while if p < Pc{m,d) it is negligible compared to the contribution of the smooth region 
ri^. For the critical exponent p = Pc{m, d) the two terms have the same order, which makes the situation 
more interesting. Note in particular that Pc(2, 2) = 2, which is consistent with our previous analysis. 

For p < Pc{m, d), we obtain the approximation rate N~"^^'^ which suggests that approximation results 
such as (|6.39p should also apply to cartoon functions and that the quantity A„i^p{f) should be finite for 
such functions. We again need to use a regularization approach, for the same reasons as in §3. For a 
given dimension d, we consider a radial nonnegative function tp of unit integral and supported in the unit 
ball of K'', and define for 5 > 



^'fi (I) fs^f* fs- 



.41) 



In order to define the quantities of involved in our conjecture, we need to introduce the second 
fundamental form of a hypersurface. At any point x € F \ be denote by n(a;) the unit normal to 
F. Note that since F is piecewise C^, the map x i-?- n{x) is on F \ V. Furthermore, we define 
TxT := n(a;)^, the tangent space to F at x. In a neighbourhood of x £ T\V, the hypersurface F admits 
a parametrization of the form 

u e T^F ^ X + U + A(u)n(x) G F \ 7^, 

where A is a scalar valued function. By definition, the second fundamental form of F at the point 
X is the quadratic form Tl^ associated to d^X{0) which is defined on T^T x T^T. Alternatively, for all 
G T^T we have Jlx{u,v) := — (9un, u). The Gauss curvature k{x) is the determinant of ILx, in any 
orthonormal basis of T^T, 

k{z) :— det 



For example, in two space dimensions the tangent space T^T is one dimensional, and we simply have 
Jlx{u, v) = k{x){u, v). We also denote by a{z) € {0, • • • , d — 1} the signature of the quadratic form II^, 
which is defined as the number of its positive eigenvalues. 
With T such that i := ^ + i, we define 

r d p' 

Spif) :=||i^(d'"/)||L.(a\r)=Ap(/|n\r). 
We conjecture the following generalization to Theorem 13.21 

Conjecture 6.2 There exists d positive constants C{k), k G {0, • • • ,d— I}, that depend on ip^p,m,d, 
such that, with 

Epif) ||C(a)|«|^[/]||i.(r) = ( ^ |CKz))|«(z)| S [/(,)] rf^) ^ 



have 

• If p < Pc then 



lim AmAfs) = Spif)- 

o— >0 



• If p — Pc then 

hm (An,p(/5)) = {SpifV + EpUr f^ . 

o— fU 



If p > Pc then 



lim (5pc lA^,p{fs) = Ep{f). 
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In the remainder of this section, we give some arguments that justify this conjecture. Given a cartoon 
function /, we define the sets ils, r°, Ts and Vs similarly to §3. We need to perform an asymptotic 
analysis of the integral 

Kid'^fsY = I Kid'^fsY + I Kid"'fsr + [ K{d'^fsy. (6.42) 
Jns Jvs Jts 



As in the proof of Theorem l3.2l the contribution of Vs can be proved to be negligible compared with those 
of Vis and Ts as 5 — > 0. The contribution of Vis satisfies 



hm / K{d"'fsY = / i^(d"7)' 
^-^^ Jn^ Jaw 



/Oa Jn\r 
The main difficulty lies again in the contribution of F^. Let us define Tc by 

1ml 

Tc ' d Pc' 

The contribution of Ts can be computed if one can establish an estimate generalizing p.25p according to 
6^K^^p{d"^fs{z)) - |[/](a;)||«;(x)|S$„ ,_^(^)(y)| < uj{5) (6.43) 

where a;(^) — as J — ?► 0, a; G Tg, u e [—1, 1], z — x+5un{x), and where the function $m.d,fc '■ [—1, 1] ^ K 
only depends on m, d, k and t^. If (|6.43l) holds, we then easily derive that 



lim<5^-i / K{d-^fsY= I C(a)|«:|^|[/]r, 



with C{k) :— J^^ \^m,d,k{u)\'^ du, which leads to the proof of the conjecture. 

We do not have a general proof of (I6.43P for any m, p and d. In the following, we justify its validity 
in two particular cases for which the explicit expression of Km,p is known to us: piecewise quadratic in 
two space dimensions {d — 2 and m = 3) and piecewise linear in any dimension (m = 2). 

Piecewise quadratic elements in two dimensions. For all 6 > 0, x G Tg and u E [—1,1], let 
'^x,s,u € P3 be the homogeneous cubic polynomial on corresponding to d^ fs{x + 5u'n{xj). Let also 
T^x,u G 1*3 be the homogeneous cubic polynomial on defined by 

7r^,„(An(a;) + ^t(.T)) = -\{<^" {u)X^ - i'^' {u)k{x)^i'^) 

for all (A, ^) e R^ where $ is defined by ((XTT)) . For aU X G F, we denote by M^.s the (symmetric) linear 
map defined by 

Mx,s'<^{x) = 5n{x) and Mx^s^{x) = ^/5t{x). 
Then, using a reasoning similar to the one used in the appendix of this paper, it can be proved that 

° Mx^s — [f]{x)TTx,u\\ < ^{S)- (6.44) 
where \m\s^QU!{6) = and the function uj depends only on /. Furthermore, it is proved in that 



if3,p(q) = C^^dis^ 

where the positive constant C depends on p and the sign of disc q. Combining this expression with (I6.44p 
proves the Estimate (I6.43|) and thus the conjecture in the case m = 3 and d = 2. 
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Piecewise linear elements in any dimension. We use the second fundamental form of the discon- 
tinuity set r in order to evaluate d"^fs on Ts. Characteristic functions are one of the simplest types of 
cartoon functions. In that case, it is possible to establish a simple relation between the second funda- 
mental form of the edge set and the second derivatives of / in a distributional sense: if C K'' is a 
bounded domain with smooth boundary T and inward normal n, we then have for all test function ip 
and u,v eR"^ 

- I dlv^= I {u,n){v,n){d^ij-ir{Jl)i^)+Jl'{u,v)i,, (6.45) 
Jn Jv 

where n^(u, v) is the second fundamental form Tlx applied to the orthogonal projection of u and v on TxT. 
The proof of this formula (that generalizes (|7.52[) which is proved in the appendix and corresponds to the 
two-dimensional case d = 2) is given further below. For all a; € F, we denote by M^^s the (symmetric) 
linear map defined by 

Mx^s'^{x) = 5n(x) and A/^. .jt — v^t 

for all t <E TxV. For all 5 > 0, x S F° and u e [—1, 1], let t:x,s,u & 1*2 be the homogeneous quadratic 
polynomial on M'^ corresponding to d'^fs{x + Sun{x)). Let also tTx,u G 1*2 be the homogeneous quadratic 
polynomial on M"* defined by 

7r:,,„(An(a;) + t) = $'(u)A2 - $(M)n^(t, t) 

for all t e TxT, where $(x) := /^d-i vix, y)dy. Then, using formula ()6.45|) and a reasoning analogous to 
the one presented in the appendix, it can be proved that 

oMx^s-m{x)TixA <^{5). (6.46) 

where lim5_>.o w((5) = and the function lj depends only on /. Furthermore, it is proved in that 

i^2,p(q) = C^|detq| 

where the positive constant C depends on d,p and the signature of q. Combining this expression with 
(j6.46|) proves the estimate (j6.43p and thus the conjecture in the case m = 2 in any dimension d > \. 

Proof of (|6.45p : Let Pr be the orthogonal projection onto F, and for all a; e F let P^; be the orthogonal 
projection onto TxT. We consider a vector u and we define U : F — > F by U(x) := Ty{x + u). If 

l|II||L~(r) < 1, then U is smooth and its differential d^; U : TxT — )• Tx'T, where 

x' = U(x), 

is given by the following formula 

4U = (Id-(u,n(a:'))n,')"'Px'- 

The determinant of dxU (more precisely the determinant of the matrix of dxU in direct orthogonal bases 
of TxT and Tx'T) is 

det(4U) det(Id-(u,n(a;'))n^,)"^(n(a;),n(x')) =1 + (m, n(x')) tr(n^O -I- \\u\\uji{u,x). 

where loi{u, x) tends uniformly to as u — > 0. Furthermore, it is easy to show that 

\i:{x + u)~ i:{x') - {u, n(x'))a„(,o^(^')l < C\\uf , 

and ||?T-(a;') — n{x) — n^/(P2;/(w))|| < ||ii||tj2(it), where C and a;2 are independent of a; € F and uJ2{u) — )■ 
as w — > 0. Combining these results, we obtain 

j ijj{x + u){n{x),v)dx — j ^{x + u){n{x),v) det{dx' TS)~^dx' 
^ J^{n{x'),vMx')dx'+J^{nix'),v){n{x'),u){dnixr)^~tTiK^^ 
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where W3(w) — t' as w — > 0. We conclude the proof of (|6.45p using the formula 

~ / i^V' = \iui h^^ / {iplx + hu) — 'ip{x)){n{x),v)dx. 
Jn ' Jr 

o 

Remark 6.3 Similarly to the results presented in there is an affine invariance property associated to 
k: if T is an affine transformation of with linear part L, and if f — f °T , T — T{T) and k is the 
Gauss curvature ofT, then one has for any s>Q, 

(detL)^ ^ |C(a)[/]n«|^ = ^ |C(a)[/]n«|^. 

It follows from this observation that when p = pc, the contribution of the edges is affine invariant in the 
sense that 

EpSf)^idetL)^EpM)- 

d- 1 

Since one also has A„j p(/) = (det L)^+t^,„ this comforts the conjecture. Let us mention that the 
quantity |k|^+t has been used in '21] in order to define surface smoothing operators that are invariant 
under affine change of coordinates. 

7 Conclusion 

In this paper we have investigated the quantity Ap(f) which governs the rate of approximation by 
anisotropic Pi finite elements as a way to describe anisotropic smoothness of functions. This quantity 
is not a semi-norm due to the presence of the non-linear quantity det(c?^/) and cannot be defined in a 
straightforward manner for general distributions. We nevertheless have shown that this quantity can be 
defined for cartoon images with geometrically smooth edges when p < 2. A theoretical issue remains to 
give a satisfactory meaning to the full class of functions for which this quantity is finite. 

From a more applied perspective, it could be interesting to investigate the role of Ap{f) in problems 
where anisotropic features naturally arise: 

1. Approximation of PDE's: in the case of one dimensional hyperbolic conservation laws, it was proved 
in that despite the appearance of discontinuities the solution has high order smoothness in Besov 
spaces that govern the rate of adaptive approximation by piecewise polynomials. A natural question 
is to ask wether similar results hold in higher dimension, which corresponds to understanding if 
Ap(f) remains bounded despite the appearance of shocks. 

2. Image processing: as illustrated in §5, the quantity Ap(f) can easily be discretizcd and defined for 
pixclized images. It is therefore tempting to use A2{f) in a similar way as the total variation in 
(|1.2p , by solving a problem of the form 

min {A2(g); \\Tg-hU.<e}, (7.47) 

geBV 

with the objective of promoting images with piecewise smooth edges. The main difficulty is that 
A2 is not a convex functional. One way to solve this difficulty could be to reformulate (|7.47p in a 
Bayesian framework as the search of a maximum of an a-posteriori probability distribution (MAP) 
as an estimator of /. In this framework, we may instead search for a minimal mean-square error 
estimator (MMSE), and this search can be implemented by stochastic algorithms which does not 
require the convexity of A2, see |18j . 
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Appendix: proof of the estimates ( 137221 ) -( 13:231 )- ( 137241 ). 



It is known since the work of Whitney on extension theorems (see in particular [52]) that for any open 
set U C M."^, and any g £ C^{U) there exists g S C^(R'^) such that g\ij = g. It follows that for each 
1 < i < k, there exists fi G C^(M^), compactly supported, and such that = fi. 

Let Tj be one of the pieces of F, between the domains flk and fli, and let s = fk and t ~ fi — fk- 
Although the domains flk and fli are only piecewise smooth, there exists an open set fl' with C'^ boundary 
such that for Jo > small enough 

f^sXn'+t on IJ {Tjj + Bs), 

0<S<So 

where Bs is the ball of radius i5 centered at 0. Note that Tj C F' d^l' and that s ~ [f] on Tj. In the 
following, the variables x, z are always subject to the restriction 



note that z e Fo a and 11 a; — z 11 < (5. We therefore have 



x E T'^ g and z — Us{x, u) — x + 6un{x) where < S < Sq and |u| < 1, (7.48) 

fs{z) ^ / s{x)(ps(z - x)dx + ts{z), 

where tg := t * ifs- The second derivatives of ts are uniformly bounded, and are therefore negligible in 
regard of all three estimates (|3.22l) , p.23p and (|3.24l) , indeed 

\\d'^ts\\L=° = Wid^t) * (ps\\l=° < \\d'^t\\L'^\\ips\\L^ = ||d^t||Loo||^||ii < oo. 

We now define the 2x2 symmetric matrices 



I{z,x) :— / (s(i) — s(.t)) d ips{z — x) dx and J{z) -.— Id (ps{z — x)dx 

so that 

d^fsiz) = dhs + I{z, x) + [f]{x)J{z). (7.49) 

We already know that the contribution of d^tg is negligible. We now prove that the same holds for the 
contribution of I{z,x). Since ips{z — x) is non-zero only if \\x — z\\ < S and therefore Hi — a;|| < 2(5, we 
can bound the norm of the matrix I{z, x) by 

\\I{z,x)\\ < 2d\\ds\\L^\\d^ips\\L^ < 2S\\ds\\L^\\d^^\\LiS-^ = C5-\ (7.50) 

This proves that the contribution of /(z, x) is negligible for the two estimates (|3.22l) and (|3.23l) . In order 
to prove that it is also negligible in the estimate (|3.24p . we need a finer analysis of t(x)'^/(z, a;)t(a;). For 
this purpose we fix a unit vector u and the pair {x, z). We introduce 

A(i) := (s(i) - s{x)) du(ps{z - x) + dus{x) ips{z - x), 

so that by Leibniz rule 

(s(i) - s{x)) dl (ps{z - £) = dl s{x) ips{z - x) - 5„A(i). 



Therefore 



u^I{z, x)u = / (9^.„s(i) ips{z — x) ~ 9„A(i)) dx, 



w 

du,u^{^) Vs{z — x)dx — I K{x){n{x),u)dx. 
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The first integral clearly satisfies 



9^,„s(i) (ps{z ~ x)dx 



< \\d^ 



and is therefore bounded independently of 6. We estimate the second integral for the special case u — t{x), 
remarking that |(n(i),t(a:))| < Ci6 on the domain of integration. Therefore 



A{x){n{x), t{x))dx 



<Ci<5|r'ni?(z,,5)|||A|Uo 



where, slightly abusing notations, we denote by \r'riB{z, 6) \ the length (1-dimensional Hausdorff measure) 
of the curve T'nB{z,S). Clearly A(i) = if > S. If \\z - S:\\ < S we have 



|A(i)| < (||a; - z\\ + \\z~ £\\)\\ds\\L^\\d^\\L^6-^ + \\ds\\L^M\L^S-^ < CqS-^ 
Since in addition |r' n B{z, S)\ < C2(5, we finally find that 



(7.51) 



A{x){n{x),t{x))dx 



< C0C1C2. 



We have therefore proved that 



|t(a;)^/(z,a;)t(a;)| < C, 



where the constant C is independent of S, which shows that the contribution of I{z,x) is negligible in 
(13:241) . 

We now analyze the contribution the quantity [/] J(z) in (|7.49p . For this purpose, we use an expression 
of the second derivative of the characteristic function Xn' of a smooth set fl' in the distribution sense. 
We assume without loss of generality that V is parametrized in the trigonometric sense, and therefore 
that n is the inward normal to f2. For all test function ip, we have 

dl^v^J = J duip{v,n) = j (a„-0(u,n) +9t'0(",t))(u,n) 

and, by integration by parts, 

at-0(u,t)(t;, n) = - / {{u,Kn){v,n) - {u,t){v,Kt)). 



Therefore, we have 



(7.52) 



(u, n) (u, n) {dnip — Kip) + k{u, t) {v, t)'ip. 
Applying this formula to ijjix) :— ips{z — x) we obtain 

Since Tj is C^, there exists a constant Co such that for all xi,X2 G Fj, we have 

|(t(a;i),n(a;2))| < Co||xi - X2\\, 

and 

|1 - (n(xi), 11(2:2)) I - |1 - (t(xi),t(a;2))| < Co\\xi - X2f. 

We finally remark that |F'ni3(z, S)\ < CiS, and that ||iy95||Loo < ||i^||ioo(5^2 and ||9n<y55||L=° < ||d(^||L~'5^^. 

Taking the vectors t{x) or n(x) as possible values of u and v in ()7.53p and using the above remarks, 
we obtain the estimates 



n{x)'^ J{z)n{x) + / dnifisiz — x)dx 



< cs- 



|t(a;)'^J(z)n(x)| < CS-\ 



t{x)'^ J{z)t{x) + / n{x)ips{z ~ x)dx 



(7.54) 
(7.55) 
(7.56) 
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where the constant C depends only on /. In view of (|7.49p We can immediately derive estimate p.23|) 
from (fTSSl) . 

In order to derive the estimate (|3.24p from (|7.56|) . we first introduce the modulus w of continuity of 
K on Tj, 

uj{S) := sup \k[xi) — k(x2)\- 

xi,X2£rj ; ||a;i— a;2||<5 

Therefore 



t(x) J{z)t{x)+ I k{x)lps{z ~ x)dx 



We now claim that 



{x)"^ J {z)t{x) + n{x) j fs{z — x)da 
tfs{z — x)dx — 5^^^{u) 



< C, 



(7.57) 



holds with C independent of 5 which implies the validity of p.24p . In order to prove ()7.57p . we use a local 
parametrization of F': let A : M — > R be such that for h small enough we have, x + ht{x) + A(/i)n(a;) G F. 
Note that we have |A(/i)| < Cq/i^ and |A'(/i)| < Co/i for h small enough. Then for 5 small enough, 



ips{z — x)dx ~ 5 ^<^{u) 



< 



ips{ht{x) + {5u - X{h))n{x))Vl + y^dh - 



ips{ht{x) + 5uxi{x))dh 



< CSiW'psWL^i^l + (Co^)2 - 1) + \\d^s\\L^Co6^) < C 
Finally, we can derive the estimate p.22p from (|7.54p using the inequality 

dnVs{z-x)di + 5-'^^'{u) <C5-^ 



(7.58) 



which proof is very similar to the one of (|7.57p . using that ||n(a;i) — n(a;2)|| < C||xi — a;2|| for all Xi,X2 £ Tj 
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